Continuity plan: BC1–BC6 - #71

services/readiness.py checks the database, file storage (a probe byte
written and deleted), Redis when limits are shared through it, and
whether the cut-off sweep has run within its interval; /ready answers
ready, degraded (Redis or the sweep: the instance still serves) or
unavailable with a 503 (the database or storage), and the metrics gain
openclear_component_up{component=…}. /health stays the load balancer's
one-question check. The runbook says what to alert on (CONTINUITY_PLAN.md BC-D6).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

The host's backup job leaves a manifest (backup_manifests/latest.json;
the runbook gives its shape) and the Controls page's "Backups fresh"
(control OC-CP-03) reads it, warning when there's none or the last backup
is older than OPENCLEAR_BACKUP_MAX_AGE_HOURS. A SQLite installation has no
backup job, so housekeeping makes a consistent copy nightly (VACUUM INTO),
keeps OPENCLEAR_SQLITE_BACKUP_KEEP_DAYS of them, and writes the manifest
itself (CONTINUITY_PLAN.md BC-D2, BC-D8). storage.read_key reads a file
OpenClear didn't write. The outage notice is excluded from the
notification preferences page, which enumerates every type (BC1's suite
found the gap).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

scripts/restore_drill.py restores a backup into a scratch database (a
copy of a SQLite file, or pg_restore), migrates it to head, verifies every
bank's action-log chain, reconciles the restored rows against file
storage, checks the dump's hash against the manifest, and records the
signed result as a restore_test row (migration e5f9a3b7d2c8); the newest
passed drill is the last restore test on every bank's Controls page, in
place of a date typed in. services/storage_reconcile.py walks every row
that names a file and every storage area (storage gains exists, ref and
list_keys on each backend): nightly as "Files and rows agree" (control
OC-CP-04, warning only) and from Platform → Continuity, which also shows
the backup manifest and the drills (CONTINUITY_PLAN.md BC-D2, BC-D3).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…n plan

OPENCLEAR_MAINTENANCE_MODE=1: people sign in and look, an administrator
can take a data export, every other change answers 503 with the reason,
every page carries the notice, and the cut-off sweep pauses, marking when
it did so the pause counts as an outage across a cut-off when the mode
ends (continuity_service.note_paused / gap_before). The Controls page
shows the mode (check maintenance_mode).

services/continuity_snapshot.py writes each bank's open and pending items
hourly as a signed CSV in its storage, keeps a dated copy at the bank's
end of day, and emails it to the incident contacts when the bank asks
(tenant.continuity_snapshot_emailed, migration f7a1b5c9d3e2); Settings →
Continuity downloads them. It's what CUEC B-CP-2-01's plan works from when
OpenClear can't be reached (CONTINUITY_PLAN.md BC-D4, BC-D5).

Tests: the export, decision-file and audit-archive areas now live under
each test's temp directory like the others, so the nightly consistency
check no longer counts the repository's own data folders.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Nine steps in the order each needs the one before: the banks told; the
keys from the key store before the database; maintenance mode on; the
restore and alembic upgrade head; file storage to the same point; /ready;
the action-log chains and the files-and-rows reconciliation; maintenance
mode off and what the first sweep then does with the gap; closing the
incident and recording the drill (CONTINUITY_PLAN.md BC-D7).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…at go through the leader lock

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…found it; the SQLite drill test skips there

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>